Implementation of data triangulation and dashboard development for COVID-19 vaccine adverse event following immunisation (AEFI) data in Nigeria.

Nigeria began administering COVID-19 vaccines on 5 March 2021 and is working towards the WHO's African regional goal to fully vaccinate 70% of their eligible population by December 2022. Nigeria's COVID-19 vaccination information system includes a surveillance system for COVID-19 adverse events following immunisation (AEFI), but as of April 2021, AEFI data were being collected and managed by multiple groups and lacked routine analysis and use for action. To fill this gap in COVID-19 vaccine safety monitoring, between April 2021 and June 2022, the US Centers for Disease Control and Prevention, in collaboration with other implementing partners led by the Institute of Human Virology Nigeria, supported the Government of Nigeria to triangulate existing COVID-19 AEFI data. This paper describes the process of implementing published draft guidelines for data triangulation for COVID-19 AEFI data in Nigeria. Here, we focus on the process of implementing data triangulation rather than analysing the results and impacts of triangulation. Work began by mapping the flow of COVID-19 AEFI data, engaging stakeholders and building a data management system to intake and store all shared data. These datasets were used to create an online dashboard with key indicators selected based on existing WHO guidelines and national guidance. The dashboard went through an iterative review before dissemination to stakeholders. This case study highlights a successful example of implementing data triangulation for rapid use of AEFI data for decision-making and emphasises the importance of stakeholder engagement and strong data governance structures to make data triangulation successful.

Lessons learned from early implementation of the Growing Expertise in E-health Knowledge and Skills (GEEKS) program in Nigeria, 2019 - 2021.

Introduction: the Growing Expertise in E-health Knowledge and Skills (GEEKS) program is an applied apprenticeship program that aims to improve informatics capacity at various levels of the national health system and create a sustainable informatics workforce. Nigeria adapted the GEEKS model in 2019 as a mechanism to strengthen data quality and use of routine immunization (RI) and vaccine-preventable disease (VPD) surveillance data among Expanded Programme on Immunization (EPI) staff. Since the start of the GEEKS-EPI program, there has not been a formal assessment conducted to measure the extent to which GEEKS-EPI has been able to build local informatics workforce capacity and strengthen RI and VPD surveillance (VPDS) data quality and use in Nigeria. Methods: we conducted a qualitative assessment to inform the extent to which GEEKS-EPI has been able to build informatics skillsets to enhance local workforce capacity, foster collaboration across government agencies, and create a sustainable informatics workforce in Nigeria. In-Depth Interviews (IDIs) and Focus Group Discussions (FGDs) were held with GEEKS-EPI supervisors, mentors, and mentees from previous GEEKS-EPI cohorts. Results: while there were challenges reported during early implementation of the GEEKS-EPI program in Nigeria, particularly early on in the COVID-19 pandemic, participants and supervisors reported that the fellowship provided a framework for building a sustainable RI and VPDS informatics workforce through regular mentorship, peer-to-peer exchanges and Subject Matter Expert (SME)-led trainings. Conclusion: lessons learned from early implementation of GEEKS-EPI in Nigeria will help to inform its implementation in other countries, where strengthened national RI and VPDS informatics capacity is the primary objective.

CDC's COVID-19 International Vaccine Implementation and Evaluation Program and Lessons from Earlier Vaccine Introductions.

The US Centers for Disease Control and Prevention (CDC) supports international partners in introducing vaccines, including those against SARS-CoV-2 virus. CDC contributes to the development of global technical tools, guidance, and policy for COVID-19 vaccination and has established its COVID-19 International Vaccine Implementation and Evaluation (CIVIE) program. CIVIE supports ministries of health and their partner organizations in developing or strengthening their national capacities for the planning, implementation, and evaluation of COVID-19 vaccination programs. CIVIE's 7 priority areas for country-specific technical assistance are vaccine policy development, program planning, vaccine confidence and demand, data management and use, workforce development, vaccine safety, and evaluation. We discuss CDC's work on global COVID-19 vaccine implementation, including priorities, challenges, opportunities, and applicable lessons learned from prior experiences with Ebola, influenza, and meningococcal serogroup A conjugate vaccine introductions.

Improving routine immunization data quality using daily short message system reporting platform: An experience from Nasarawa state, Nigeria.

Routine immunization (RI) delivery was declared a public health concern in Nigeria in 2017 because of persistently low immunization coverage rates reported in independent surveys. However, administrative coverage rates remain high, suggesting serious data quality issues. We posit that a shorter timespan between service provision and data reporting can improve the monitoring of RI data, and developed a short message system (SMS) text reporting strategy to generate daily RI data points from health facilities (HFs). The goal was to assess whether daily data collection produces complete, reliable and internally consistent data points. The SMS reporting platform was piloted between December 2017 and April 2018 in two Local Government Areas (LGAs, equivalent to districts) of Nasarawa state, Nigeria. The 145 healthcare workers from 55 HFs received one mobile phone and pre-configured SIM card, and were trained to send data through predefined codes. Healthcare workers compiled the data after each vaccination session and transmitted them via SMS. We analyzed completeness, number of weekly sessions, and supportive supervision conducted. During the pilot phase, we received data from 85% (n = 47) of the 55 HFs. We expected 66 fixed-post sessions and 30 outreach sessions per week, but received data for 33 fixed-post and 8 outreach weekly session on average. More HFs reported on Tuesdays compared to other days of the week. When assessing internal consistency, we observed that the reported number of children vaccinated was sometimes higher than the number of doses available from opening a given number of vaccine vials. When found, this discrepancy was noted for all antigens during fixed-post and outreach vaccination sessions. Despite these initial discrepancies, transmitting RI data sessions via texting is feasible and can provide real-time updates to the performance of the RI services at the HF level.

Pilot implementation of a routine immunization module of the district health information system version 2 in Kano State, Nigeria, 2014 - 2015.

Introduction: Timely and accurate data are necessary for informing sound decision-making and developing effective routine immunization (RI) programs. We launched a pilot project in Kano State to strengthen routine immunization (RI) data reporting through the immunization module of the District Health Information System version 2 (DHIS2). We examined the completeness and timeliness of reporting monthly RI data one year before and one year after DHIS2 module pilot in the State. Methods: The first phase of the DHIS2 RI module pilot in Kano included training on RI data tools in November 2014 and in January 2015 for 36 state and zonal personnels, 276 local government area (LGA) personnel, and 2,423 health facility (HF) staff. A RI-focused dashboard to display core RI accountability framework indicators, such as completeness and timeliness of reporting, planned immunization sessions conducted, coverage and dropout was implemented. Report completeness was ratio of submitted reports to number of health facilities while report timeliness was ratio of reports on the DHIS2 by 14th of the month to number of expected. Results: Completeness of data reporting increase from 70% in 2014 to 87% in 2015, while timeliness of reporting increase from 64% to 87% over the same period. Challenges encountered during the implementation process included limited access to internet, power outages, health workers strike, staff attrition and competing state activities. Conclusion: The pilot implementation of the DHIS2 immunization module in Kano State led to modest improvement in the reporting of RI services. Several lessons learned were used to guide scale-up to other states in the country.

Data management needs assessment for the scale-up of district health information system and introduction of routine (essential) immunization module in Bauchi State, Nigeria, 2015.

Introduction: the National Primary Health Care Development Agency, African Field Epidemiology Network, United States Centers for Disease Control and Prevention and the Bill and Melinda Gates Foundation are implementing a Routine Immunization (RI) Module as part of their Routine Health Data Management System based on the 2013 - 2015 Accountability Framework for RI in Nigeria. To inform planning and evidence-based decision making, a data management needs assessment was conducted in Bauchi state which was one of the states selected for the deployment of the DHIS2 RI module. Methods: desk reviews were conducted, and a semi-structured questionnaire was administered in four Local Government Areas (LGAs) in Bauchi state that were selected based on the initial evaluation of the performance of all 20 Bauchi LGAs. Ganjuwa and Shira were selected as high-performing LGAs and Alkaleri and Bogoro as low-performing LGAs. Four Health Facilities (HF) were selected in each LGA based on rural or urban classification, type of HFs (private or public), security and accessibility. Results: local Immunization Officers (LIOs) prepare monthly reports in high-performing LGAs, and Community Health Care workers are mostly (69%) responsible for report compilation at the HFs. Shira and Alkaleri met 77% and 44% of training indicator targets, respectively, in the previous 12 months. Data recording and reporting was the type of training received the most by health facility personnel. Functioning refrigerators were in all visited LGAs, working thermometer and updated temperature monitoring charts were available in all the cold chain stores. However, no health facility reported having available computers for data-related activities. Conclusion: this assessment provided an improved understanding of the Bauchi state Routine Health Data Management System and informed the content of the state-wide scale-up.

Use of a district health information system 2 routine immunization dashboard for immunization program monitoring and decision making, Kano State, Nigeria.

Introduction: a district health information system 2 tool with a customized routine immunization (RI) module and indicator dashboard was introduced in Kano State, Nigeria, in November 2014 to improve data management and analysis of RI services. We assessed the use of the module for program monitoring and decision-making, as well as the enabling factors and barriers to data collection and use. Methods: a mixed-methods approach was used to assess user experience with the RI data module and dashboard, including 1) a semi-structured survey questionnaire administered at 60 health facilities administering vaccinations and 2) focus group discussions and 16 in-depth interviews conducted with immunization program staff members at the local government area (LGA) and state levels. Results: in health facilities, a RI monitoring chart was used to review progress toward meeting vaccination coverage targets. At the LGA, staff members used RI dashboard data to prioritize health facilities for additional support. At the State level, immunization program staff members use RI data to make policy decisions. They viewed the provision of real-time data through the RI dashboard as a "game changer". Use of immunization data is facilitated through review meetings and supportive supervision visits. Barriers to data use among LGA staff members included inadequate understanding of the data collection tools and computer illiteracy. Conclusion: the routine immunization data dashboard facilitated access to and use of data for decision-making at the LGA, State and national levels, however, use at the health facility level remains limited. Ongoing data review meetings and training on computer skills and data collection tools are recommended.

Strengthening facility-based immunization service delivery in local government areas at high risk for polio in Northern Nigeria, 2014-2015.

Introduction: The National Stop Transmission of Polio (NSTOP) program was created in 2012 to support the Polio Eradication Initiative (PEI) in Local Government Areas (LGAs) at high risk for polio in Northern Nigeria. We assessed immunization service delivery prior to the commencement of NSTOP support in 2014 and after one year of implementation in 2015 to measure changes in the implementation of key facility-based Routine Immunization (RI) components. Methods: The pre- and post-assessment was conducted in selected health facilities (HFs) in 61 LGAs supported by NSTOP in 5 states. A standardized questionnaire was administered to the LGA and HF immunization staff by trained interviewers on key RI service delivery components. Results: At the LGA level, an increase was observed in key components including availability of updated Reach Every Ward (REW) micro-plans with identification of hard to reach settlements (65.6% baseline, 96.8% follow-up, PR = 1.5 (95% CI 3.4 - 69.8), vaccine forecasting (77.1% baseline, 93.5% follow-up, PR =1.2 (95% CI 1.8 - 13.8), and timely delivery of monthly immunization reports (73.8% baseline, 90.2% follow-up; PR =1.2 (95% CI 1.2 - 9.0). At the HF level, there was an increase in percentage of HFs with written supervisory feedback (44.5% baseline, 82.5% follow-up, PR = 1.8 (95% CI 4.7 - 7.3), written stock records (66.5% baseline, 87.9% follow-up, PR = 1.3 (95% CI 2.9 - 4.7) and updated immunization monitoring charts (76.3% baseline, 95.6% follow-up, PR = 1.3 (95% CI 4.6 - 9.9). Conclusion: We observed an improvement in key RI service delivery components following implementation of NSTOP program activities in supported LGAs.

Targeted Short Message Service-Based Intervention to Improve Routine Immunization Reporting in Bauchi State, Nigeria, 2016.

Introduction: High quality, timely and complete immunization data are essential for program planning and decision-making. In Nigeria, the National Health Management Information System (NHMIS) Routine Immunization (RI) module and dashboard (on the District Health Information System version 2 (DHIS2) platform) support the use of real time RI data. We deployed an automated short message service (SMS) notification system that works with the existing RI module to facilitate improvements in RI data in the DHIS2. Methods: A pilot project was performed using intervention and control local government areas (LGAs). A mixed methods approach using both qualitative and quantitative methods was used to evaluate the system. We assessed changes in reporting rates across different reports. The evaluation also included baseline and post-intervention surveys of health facility (HF) staff. Results: Reporting timeliness (76% pre and 99% post intervention) and completeness (83% pre and 99% post intervention) were consistently higher during the post-intervention than the pre-intervention period for facilities in the intervention LGA while reporting timeliness (65% pre and 66% post intervention) and completeness (71% and 77% post intervention) for facilities in the control LGA showed no change. Users reported that the SMS system was easy to understand and helped to facilitate improvements in consistency of data and timeliness of reporting. Inability of health care workers to effect changes at the HF level and the lack of immediate feedback were reported as key challenges to timeliness and quality of reports. Conclusion: An SMS-based intervention improved timeliness and completeness of health data reporting. However, the intervention should be evaluated on a larger scale over a longer time period before considering a national implementation.

Determinants of improved data consistency across routine immunization data tools for health facilities in Kano State, Nigeria.

INTRODUCTION: in this study, determinants of improved data consistency for routine immunization information at health facilities was measured to identify associated factors. METHODS: between June and August 2015, 1055 HFs were visited across 44 Local Government Areas in Kano state. We assessed data consistency, frequency of supportive supervision visits, availability of trained staff and attendance to monthly LGA RI review meetings. We compared RI monthly summary forms (MSF) versus national health management information system summary form (NHMIS) and vaccine management form 1a (VM1a) versus HF vaccine utilization summary monthly summary (HFVUM) for consistency. Data consistency at HF was determined at <+10% between number of children reportedly immunized, and doses of vaccine opened using 3 antigens (BCG, Penta and Measles). Levels of discrepancy <10% were considered as good data consistency. Bivariate and multivariate analysis used to determine association. RESULTS: data Consistency was observed in 195 (18.5%) HFs between (MSF vs NHMIS) and 90 (8.5%) HFs between (VM1a vs HFVUM). Consistency between MSF vs NHMIS was associated with receiving one or more SS visits in the previous month (p=0.001), data collection tools availability (p=0.001), recent attendance to monthly LGA RI review meeting and availability of trained staff. Data consistency between VM1a form and the HF VU summary was associated with a recent documented SS visit (p=0.05) and availability of trained staff (p=0.05). CONCLUSION: low level of data consistency was observed in Kano. Enhanced SS visits and availability of trained staff are associated with improved data quality.

Implementing the routine immunisation data module and dashboard of DHIS2 in Nigeria, 2014-2019.

In 2010, Nigeria adopted the use of web-based software District Health Information System, V.2 (DHIS2) as the platform for the National Health Management Information System. The platform supports real-time data reporting and promotes government ownership and accountability. To strengthen its routine immunisation (RI) component, the US Centers for Disease Control and Prevention (CDC) through its implementing partner, the African Field Epidemiology Network-National Stop Transmission of Polio, in collaboration with the Government of Nigeria, developed the RI module and dashboard and piloted it in Kano state in 2014. The module was scaled up nationally over the next 4 years with funding from the Bill & Melinda Gates Foundation and CDC. One implementation officer was deployed per state for 2 years to support operations. Over 60 000 RI healthcare workers were trained on data collection, entry and interpretation and each local immunisation officer in the 774 local government areas (LGAs) received a laptop and stock of RI paper data tools. Templates for national-level and state-level RI bulletins and LGA quarterly performance tools were developed to promote real-time data use for feedback and decision making, and enhance the performance of RI services. By December 2017, the DHIS2 RI module had been rolled out in all 36 states and the Federal Capital Territory, and all states now report their RI data through the RI Module. All states identified at least one government DHIS2 focal person for oversight of the system's reporting and management operations. Government officials routinely collect RI data and use them to improve RI vaccination coverage. This article describes the implementation process-including planning and implementation activities, achievements, lessons learnt, challenges and innovative solutions-and reports the achievements in improving timeliness and completeness rates.

Strengthening the Acute Flaccid Paralysis (AFP) Surveillance Component of the Polio Eradication Initiative through Short Message Service (SMS) Reminders; Experience from Sokoto State, Nigeria 2014.

Eradication of poliomyelitis remains a public health priority due to the paralytic effects of the virus on children and impact on global health system. However, existing gaps in surveillance can hinder eradication. Improved timeliness of identification and reporting of acute flaccid paralysis (AFP) cases with further confirmation of Wild Poliovirus (WPV) in stool samples, can help Nigeria achieve the performance indicators of non-polio AFP rate of ≥ 2/100,000 population aged < 15 years and ≥80% stool sample collection adequacy. To ascertain the awareness of AFP case definition and detection by health care workers and to evaluate the impact of SMS-based reporting on the AFP surveillance system the study was conducted from November 2013 to July 2014. In Sokoto state, 112 health facilities (focal sites) were operational and participated in this study. All AFP focal points for the 112 facilities were included in the study. In addition to AFP focal points, two clinicians per facility where possible, were included in the study. The study focused exclusively on reports from focal sites. The methodology was a one group pretest-posttest design conducted in 3 phases. 1) Pre-intervention Knowledge, Attitude and Practices (KAP) survey, 2) SMS implementation and 3) Post-intervention KAP. Results were analysed using the independent sample t-test to assess the increase in knowledge, attitudes, or practice scores pre- and post- training. The study showed improved knowledge gap of health care workers on AFP surveillance between pre and post intervention. It shows that this approach of improved surveillance will be effective in countries in hard to reach, access compromised or countries/place without sufficient surveillance staff.